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The  paper  considers  nonlinear  time  series  whose  second  order  autocorrelations  satisfy  auto 
regressive  Yule-Walker  equations.  T^ie  usual  linear  residuals  are  then  uncorrelated,  but  not 
independent,  as  v/ould  be  the  case  for  linear  autoregressive  processes.  Two  such  types  of  non¬ 
linear  model  are  treated  in  some  detail:  random  coefficient  autoregression  and  multiplicative 
autoregression.  The  proposed  analysis  involves  crosscorrelation  of  the  usual  linear  residuals 
and  their  squares.  This  function  is  obtained  for  the  two  types  of  model  considered,  and  allows 
differentiation  between  models  with  the  same  autocorrelation  structure  in  the  same  class.  For 
the  random  coefficient  models  it  is  shown  that  one  side  of  the  crosscorrelation  function  is 
zero,  giving  a  useful  signature  of  these  processes.  The  non-zero  features  of  the  other  side  of 
the  crosscorrelations  are  informative  of  the  higher  order  dependency  structure.  In  applications 
this  residual  analysis  requires  only  standard  statist^ical  calculations,  and  extends  rather  than 
replaces  the  usual  second  order  analysis.  ) 
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1.  ZNTRODOCnOII 


Ttiis  pap«r  is  concsmsd  with  ths  statistical  asssssmsnt  of  dspendency 
bsyond  autocorrslation  in  ths  context  of  nonlinear  tine  series  models.  The 
central  theme  is  that  residuals  from  models  fitted  according  to 
inapplicable  linearity  assumptions  can  also  profitably  be  used  for  further 
analysis  beyond  linearity.  A  well  known  property  of  linear  time  series 
residuals  is  that  they  are  uncorrelated;  in  the  context  of  lineau:  models 
they  should  also  be  independent,  apart  from  the  effects  of  parameter 
estimation  with  short  series.  However,  in  the  context  of  nonlineair  models 
it  is  not  often  recognised  that  uncorrelated  residuals  also  hold 
information  concerning  higher  order  dependence  in  the  data.  Developments 
of  such  a  higher  order  residual  analysis  will  be  explored  here,  in 
particular  for  two  types  of  nonlinear  autoregressive  model  which  have  the 
usual  linear  Yule-Walker  autoregressive  correlation  structure.  Some  higher 
order  dependency  correlations  will  be  obtained.  In  a  later  paper,  the 
suggestion  of  reversed  residuals  will  be  made,  and  the  analysis  given  in 
this  paper  will  be  extended  to  encompass  these  reversed  residua'ls. 

2.  AUTOREGRESSION  AND  IiXNEAR  AUTOREOtESSXVE  RESIDUALS 
2.1  Autoregression  for  Nonlinear  Models 

The  standard  form  of  autoregression  needs  widening  for  use  with 
nonlinear  models;  we  consider  first  the  standard  form  which  is  explicitly 
autoregressive  in  a  linear  additive  way  and  then  several  weaker  variants. 
A  stationary  time  series  (X^}  of  mean  n  is  assumed. 

Under  the  linear  autoregressive  model,  of  order  p,  the  {X^}  satisfy  the 
equation 

X^-ji-ai(  Xt-i-;i  )+a2(  X^-2-M  )+•  •  •+<*p(  Xt-p-4  )+«t'  t»0,  ±1,  ±2 ,  .  .  .  (2.1) 

where  the  e^  are  independent  and  identically  distributed  and  ^.^1^02, • . . ,op 
are  fixed  parameters;  these  parameters  are  also  assumed  to  satisfy  the 
condition  that  the  polynomial  1-aiz-. . . -OpsP  has  all  roots  outside  the  unit 


circle . 


A  more  general  definition  of  autoregression  of  order  p,  could  be  the 
linmar  conditional  mxpoetation  requirement  that 

•  ^  *  t*0  i  ^1 1  tZ  #•••  (2*2) 

Note  that  the  conditional  expectation  is  with  respect  to  all  previous  Xf 
The  definition  (2.1)  is^lies  (2.2)  but  not  vice-vem.  Thus  this 
definition  could  apply  to  models  %diich  are  not' of  the  .near  form  (2.1), 
either  because  the  e^  are  dependent,  but  still  with  «.(  IXt-i/Xt-2'  •  •  •  )~0, 
or  because  the  model  has  some  other  structure  altog^  ;.  For  instance, 
there  ue  the  random  coefficient  models  of  Nicholls  a  Quinn  (1982),  the 
exponential  distribution  random  coefficient  models  of  •awrance  and  Lewis 
(1981,1985),  the  discrete  distribution  random  coefficient  inod«.\s  of  Jacobs 
and  Lewis  (1983),  and  the  gamma-beta  random  coefficient  models  of  Lewis 
(1981 ). 

Random  coefficient  autoregressive  models  of  order  p  take  the  general 

form 

Xt  -  Xt-i+At(2>Xt-2+.  .  .■♦•At<P>Xt-p+Bt,  t-0,  tl,  *2,  ,  .  .  (2.3) 

where  the  vector  of  coefficients  (A^^ is  a  stationary 
vector  sequence  of  independent  random  variables,  independent  of 

Xr.X'^-2' ■  ■  ' '  sometimes  in  addition,  is  independent  of  the  random 

coefficients  A^(^)  at  time  t.  A  roots  condition,  analogous  to  that 
required  for  (2.1)  is  also  needed.  Zt  is  easy  to  see  that  such  models 

satisfy  the  linear  conditional  expectation  definition  of  pth  order 
autoregression,  but  are  nonlinear  in  the  sense  of  (2.1);  see  also  Lawrance 
and  Lewis  (1985)  for  further  discussion  of  the  meaning  of  nonlinearity.  A 
first  order  case  of  the  type  (2.3)  will  be  used  to  illustrate  the  proposed 
method  of  residual  analysis  of  autoregressive  nonlinearity. 

A  further  and  weaker  definition  of  autoregression  is  the  requirement 

that  the  autocovariances  of  the  (X^),  denoted  by  ('yy}*  just  satisfy 

Yule-Walker  linear  difference  equations  of  the  form, 
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yr^iyr-i+a2yr-2+*  •  •+<*pyt-p»  Yr^-v  <2. 4) 

for  suitabl*  constants  a^,a2, .  •  •  ,(Xpt  this  will  bs  rsferrsd  to  as 
Yul«-Walk»r  autor»gr»»»ion.  It  is  trus  for  procsssas  which  satisfy  (2.1) 
and  (2.2),  as  may  bs  vsrifisd  in  ths  tisual  mannsr  by  multiplying  Xt  by  X^-r 
and  talcing  sj^sctations .  Ihs  rsvsrss  is  not  txus>  (2.4)  doss  not  in^ly 
(2.1)  or  (2.2).  A  cass  in  which  (2.4)  holds  but  in  Which  (2.1)  emd  (2.2) 
do  not,  is  ths  product  autorsgrsssion  modsl  of  McKsnzis  (1982),  whsrs,  with 
p>l,  E(Xt  |X^-l,Xt-2> .  • .  )  ia  4  fractional  powsr  of  X^-i;  this  modsl  will 
also  bs  ussd  in  Ssction  4.3  as  an  ilustration  of  ths  propossd  rssidual 
msthods . 

A  varisty  of  othsr  modsls  can  satisfy  ths  Yuls-Walksr  dsfinition  of 
autorsgrsssion,  and  yst  not  satisfy  ths  linsar  conditional  sxpsctation 
dsfinition.  Amongst  thsss  ars  first  ordsr  Markov  chains  undsr  csrtain 
conditons,  and  othsrs  such  as  ths  ssmi-Markov  gsnsratsd  procsssss  discussed 
in  Cox  and  Lswis  (Chaptsr  7,  1966)  and  Lewis  (1980).  Notice  that  we  do  not 
define  nonlinear  autoregressive  modsls  in  a  constructive  way.  Ths  class  is 
so  wide  as  to  maks  this  impossible;  one  such  class  has  been  studied  by 
Jones  (1978).  Rather,  t#e  require  that  ths  autocorrelations  should  satisfy 
linear  equations,  similar  in  structure  to  those  satisfied  by  the 
autocorrelations  of  linear  autoregressive  models.  In  view  of  this,  our 
suggested  analysis  extends,  rather  than  supercedes,  conventional  methods. 

2.2  Definition  and  Discussion  of  Linsar  Autoregressive  Residuals 

For  ths  analysis  of  time  series  data  involving  models  satisfying  (2.1), 
or  (2.2)  or  minimally  (2.3),  the  use  of  linear  autoregressive  residuals  of 
order  p,  defined  as 

R^(  P )  -  ( X^-H  )-ai(  X^-i-u  )-a2<  Xt-2~>^  >“  •  •  •  -ap(  X^-p-U )  (2.5) 

is  suggested.  This  suggestion  is  based  on  the  following  theorem,  which  is 
a  generalization  of  a  result  given  in  Lawrence  and  Lewis  ( 1984,  Section 


IhaorMi.  Let  th«  stationary  procoss  (X^)  satisfy  ths  Yuls-Walker  type 
aquations  (2.4).  Than  tha  linaar  autoraqraasiva  rasiduals  dafinad 

at  (2.5)  ara  uncorralatad  ( although  not  nacassarily  indapandant ) . 

Proof.  ITta  autocovariancas  of  tha  rasiduals  (2.5)  ara 
Cov[Rt<P\Rt+r<^*>]  -  Cov[(Xt-*i),Rfc+r(P>]-oiCov[(Xt-i-M),Rt+r^P^]-.-. 

-«pCoa[(Xt-p-»i),Rt+r^P>]  (2.6) 

-  CovC  ( Xt-n ) ,  Rt+r^  P  >  ]-aiCovC  ( XfH ) ,  Rt+r+1^  P  >  ]- .  . . 

“apCov[(Xt-4),Rt+rV^^5-  <2.7) 

Equation  (2.7)  follows  Isacausa  the  (X^)  process  is  stationary  and 
consequently  tha  (Rt^P^}  process  is  stationary.  The  covariances  in  (2.7) 
need  only  be  considered  for  positive  lag  since  autocovariance  is  an  even 
function  of  r.  Than  tha  crosscovariancas  on  tha  right-hand  side  of  (2.7) 
are  all  of  tha  sane  type  aind  given  by 
Cov[  ( XfM )  /  Rt+r^  P  h  -  CovC  ( ) ,  {( xt+r-M  )-cii(  xt+r-l->* )  •  •  •  Xt+r-p->^ ) )  ] 

-  yr“<*iyr-l”  •  •  •  “<*pyr-p  r-1 , 2 .  (2.8) 

Now  by  tha  Yula-WaUcar  aquations  (2.4),  tha  expression  (2.8)  is  zero.  Thus 
using  (2.8)  in  (2.7) 

Cov[Rt(P),Rt+r^P>]  -  0  r-*l,t2,...,  (2.9) 

as  was  to  be  proven.  Tha  proof  is  immediate  for  tha  linear  autoregressive 

model  (2.1),  since  R^^P^  ~  et'  ^t  Rt-t-r^^^  ^Y  definition 

independent . 

Note  that  the  linaar  autoregressive  residuals  will  generally 

still  be  dependant  though  uncorralatad  in  nonlinear  modelling  of  the  (Xt); 

with  the  linear  autoregressive  model  (2.1)  the  (Rt^P^)  util  not  only  be 

uncorreiated  but  also  independent.  It  ts  this  diffsrsnee  uhich  util  be 

exploited  to  explore  nonlinearity  in  ptb  order  nonlinear  autoregressive 

processes .  The  dependency  attributable  to  parameter  estimation  is  taken  to 

be  small  in  the  large  scale  applications  we  have  in  mind;  indeed,  nonlinear 

modelling  of  short  series  of  data  may  well  be  hud  to  justify. 

The  quantities  Rt^P^  are  autoregressive  residuals  in  the  sense  that 

Rt^^^  is  the  residual  of  X^  after  subtracting  off  its  best  linear 
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l«ut  squar**  pradictor  In  taraa  of  Xt-i>Xfc-2>  •  •  •  >  givan  by 

«  »i+ai( )+02( )+ . .  •  +ap( Xfe_p-|t ) .  (2.10) 

Thus  tha  rasidu&ls  9lva  tha  basic  way  of  taking  out  the  linear 

correlation  component  in  models  with  pth  order  autoregression  minimally  of 
the  'Yule-walker'  form  (2.4). 

A  further  point  %#orth  noting  about  the  residuals  concerns  their 

crosscovariances  with  the  Xt's.  Zn  the  proof  of  the  theorem  it  is  seen 
that  Cov[Xt-»i.fRt+r^P^]  “  0  r  »  1,2,..,  ,  However,  the  other  half  of 
these  crosscovariances  is  non-zero. 

Example:  The  ( nonOaussian)  linear  AR(1)  model 

Taking  (2.1)  with  p>l,and  p  instead  of  a^,  the  crosscorrelation  of  X^-m  a^d 
Rt-r^^^  in  the  AR(1)  model  is  given  by 

Corr(Xt-U.Rt-r)  *  (1-P^)^/^  for  r-1,2,3,,,.;  (2.11) 

the  superscript  has  been  dropped  from  Rt-r*  ^  it  will  be  in  similar  future 
use.  Note  further,  that  in  this  case  Rt+r  i^  independent  of  for 
r>l,2,...  .  Further  use  of  (2.11)  will  be  made  in  Sections  3  and  5. 

For  the  use  of  in  data  analysis,  the  order  p  of  the  linear 

aspect  of  the  autoregression  needs  to  have  been  chosen;  any  of  the 
available  standard  methods  may  still  be  used.  Zn  addition,  of  course,  the 
u,ai,a<2, . . . ,Op  need  to  be  estimated;  there  are  at  least  two  convenient 
possibilities:  (1)  the  assumption  of  a  lineair  autoregressive  model  like 

(2.1)  and  the  use  of  least  squares  estimation,  emd  (2)  a  non-model  based 
approach  to  estimation  employing  the  first  p  Yule-Walker  type  equations 
(2.4).  The  latter  is  suggested  here;  however,  Tjostheim  and  Paulsen  (1963) 
recommend  (1)  when  dealing  with  modest  sized  seunples  from  linear  models,  on 
account  of  serious  estimation  bias  with  the  Yule-Walker  estimates.  Since 
the  use  envisaged  here  is  primarily  with  nonlinear  models  and  substantial 
sets  of  data,  the  superiority  of  (1)  over  (2)  is  not  established  or 


crucial. 


Sine*  the  lln«ar  autora^rMSlv*  ■  residuals  of  (2.5)  have  zero 

covariances  idien  (2.4)  holds,  but  for  nonlinear  processes  need  not  be 
independent,  a  residual  analysis  of  this  nonlinearity  can  be  based  on  em 
assessment  of  their  higher  order  dependence.  To  consider  what  form  this 
might  taka,  we  note  that  the  use  of  (Xt^}  has  been  suggested  by  Gremger  and 
Andersen  ( 1978, p. 63)  for  bilinear  models,  for  which  in  many  of  the  simple 
cases  the  have  zero  autocorrelations  themselves.  The  corresponding 
initial  suggestion  here  for  a  residual  analysis  is,  as  was  briefly 
illustrated  in  lAunrance  and  Lewis  (1985),  to  use  the  residuals 
Displays  can  then  euily  be  made  of  the  associated  autocorrelation 
functions,  scatter  plots,  periodograms ,  cumulative  periodograms ,  etc,  using 
standard  (second  order)  time  series  software. 

Hov^ever,  the  autocorrelations  of  {(Rt^^^)^}  ere  fourth  order  quantities 
in  the  original  series  (X^),  which  is  a  double  jump  from  the  second  order 
autocorrelations  of  the  series  (Xt).  Such  quantities  will  be  very 
difficult  to  handle  theoretically  with  most  types  of  nonlinear  model. 
The  croascorrmlation  function  of  and  i»  essentially 

third  order  and  is  preferred  here;  for  previous  use  see  Subba  Rao  and  Gabr 
(1984)  and  Lawrance  and  Lewis  (1965).  Since  this  crosscorrelation  function 
involves  the  variance  of  {(Rt^P^)^)/  it  also  needs  some  fourth  order  joint 
moments  of  {X^}  up  to  lag  p.  The  necessary  calculations  of  all  these 
quantities  for  two  nonlinear  models  will  be  given  in  Section  4. 

The  behaviour  of  the  crosscorrelation  function  of  (Rt^^^)  and 
((r^(P))2)  may  be  judged  against  the  fact  that  the  are  independent 

for  the  linear  autoregressive  model  of  order  p,  and  hence  the 
crosscorrelation  function  of  and  {(Rt^^M*)  will  be  zero  except  at 

lag  zero.  For  the  random  coefficient  autoregressive  processes  considered 
in  Section  4,  the  suggested  crosscorrelation  function  will  be  shown  to 
posess  a  useful  rut-off  property;  this  generatlizes  the  NEAR( 2 )  result  given 


in  Lawranc*  and  Lawla  ( 1985 ) . 

Tvo  rathar  siiq>l«r  croascorralation  quantitiaa  can  ba  proposad  from 
considaration  of  tha  zaro  covarianca  raault  of  (2.8),  and  tha  additional 
fact  that  whan  tha  pth  ordar  linaaz  autoragrassiva  modal  holds,  and 
(r-1,2,...)  will  ba  indapandant,  and  not  just  uncorrelated. 
Working  in  terms  of  {X^}  adjusted  for  its  mean  fi,  which  is  better 
con^utationally  and  often  nicer  theoretically,  the  following  quantities  may 
be  considered, 

Corr[(Xt-M)2,Rt+r^^^l'  for  r-0, ti, ±2, . . .  (3.1) 

Corr[Xt-»i,(Rt+r^P^>^l'  r-0tl,t2 .  (3.2) 


These  autocorrelations  are  not  equivalent  in  the  aspects  of  higher  order 
dependency  of  the  {X^}  process  which  they  aissess.  To  see  this,  note  that 
the  covariance  corresponding  to  (3.1)  involves  only  third  order  joint 
moments  of  the  form  E[(Xt— M)^(Xt+r-i“M)],  i“0, ±1, . . . , tp  while  the 
covariance  corresponding  to  (3.2)  involves  additional  joint  moments  o£  the 
form  ECXt-M)(Xt+r-tt)()Ct+j._i)],  i-0, *1, . . . ,  tp.  Further,  the  denominator  of 


(3.2),  by  needing  var(( )^) ,  is  more  complicated  in  the  higher  order 
moments  it  involves  relative  to  the  denominator  of  (3.1)  which  needs 
varCRt^P)).  However,  both  correlations,  and  particularly  (3.1),  are  more 
tracteUDle  than  the  autocorrelations  of  ((Rt^P^)^)  or  the  crosscorrelations 


of  (Rt^P^  and  {(Rt^P')^). 

Exao^le  (Continued  from  (2.11)):  The  ( nonGaussiam )  linear  AR(1)  model. 

As  an  illustration  of  the  use  of  (3.1)  atnd  (3.2),  for  the  linear  AR(  1 ) 


model,  ((2.1)  with  p-1  and  p  instead  of  )  there  are  the  results 


CorrC(Xt-M),(Rt-r)^]  - 


r— 1,-2,  .  . . 


[vIf(^R^)iv  >^-0,1, 


(3.3) 


Corr[(Xt-n)2,Rt-r] 


r— 1,-2, 


Var(X) 


il£L„rr 


(3.4) 


Note  the  faster  geometric  decreaise  in  (3.4),  for  non  negative  lags. 


relative  to  (3.3);  also  (3.3)  contains  more  higher  moment  information.  The 
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behaviour  of  these  functions  —  zero  for  r<0  axtd  geometric  decay  for  r>0  - 
suggests  linear  models  with  skewed  marginal  distributions  or  nonlinear 
models.  Discussion  of  parallel  results  for  two  types  of  nonlinear  model 
are  given  in  section  4. 

The  use  of  squaring  in  the  construction  of  these  higher  order 
dependency  measures  is  recognized  as  being  pragmatic  and  somewhat  arbitary; 
it  does  ho%#ever  lead  to  expressions  involving  selected  types  of  simpler 
higher  order  cross  moments .  The  end  use  of  the  higher  order  dependency 
measures  can  either  be  exploratory  for  a  given  data  set,  to  ascertain 
whether  there  is  appreciable  nonlinearity  present,  or  constructively,  to 
provide  evidence  for  fitting  suitable  types  of  nonlinear  model  which  can 
match  the  observed  higher  order  dependency. 

®**^^^*^  discussion  of  higher  order  dependence  in  nonGaussian  tinaar 
models  is  given  by  Rosenblatt  ( 1980 )  in  terms  of  the  bispectrum,  aspects  of 
nonlinearity  and  higher  order  spectra  are  briefly  considered  in  Rosenblatt 


4.  HIGBER  ORDER  DEPBIDENCT  FOR  TND  TXPES  OP  NONLINERR  M3DEI. 

In  this  section  we  obtain  properties  of  the  measures  of  higher  order 
dependency  discussed  in  the  previous  section,  for  two  specific  types  of 
nonlineeu:  models.  The  models  considered  are  autoregressive  in  the 
Yule-Walker  sense  of  (2.4),  but  not  in  the  lineau:  sense  of  (2.1). 

4.1  Randoa  Coefficient  Rutoregzeeeive  Modele 

As  already  remarked  in  Section  1,  a  generalization  of  the  linear 
autoregressive  model  (2.1)  is  to  let  the  coefficients  ai,a2, . . .Op  be  ramdoro 
variables.  One  general  class  of  such  models  has  been  discussed  by  Nicholls 
and  Quinn  ( 1982 )  who  cite  Andell  ( 1976 )  and  articles  in  the  economic 
literature.  Other  classes  of  models  with  random  coefficients  include  the 
discrete  distribution  models  of  Jacobs  and  Lewis  ( 1983 )  and  the  exponential 
models  of  Lawrance  and  Levris  (1981,1985).  The  class  of  random  coefficient 
autoregressive  processes  considered  here  is  given  by 

Xt  -  ^ )Xt-i+At( 2 )Xt-2+ . . . P  >Xt_p+Bt  t-0 , tl , 12 , . . . ,  (4.1) 

where  (At^ ^ At^ ^ ^, . . . ,At( P^,Bt )  is  independent  of  X^-i , Xt-2 » > • •  and  forms 
a  stationary  vector  sequence  of  independent  random  variables  where 
E(A|.^^))-aj  for  j“l,...,p  satisfy  the  stamdard  polynomial  equation.  The 
components  of  the  vectors  are  not  necessarily  independent .  For  example  the 
discrete  distribution  models  of  Jacobs  and  Lewis  (1983)  can  be  written  in 
this  fonn,  and  have  dependent  coefficients,  as  do  the  exponential  models  of 
Lawrance  and  lawis  (1981,  1985).  It  is  easily  verified  that  the  process 

(4.1)  satisfies  conditional  expectation  autoregression  (2.2)  and  thus  also 
the  weaker  Yule-Walker  definition  (2.4);  it  will  also  clearly  be 
stationary.  Note  as  well  that  the  standard  linear  autoregressive  model 

(2.1)  is  a  special  case  of  (4.1)  in  which  the  random  coefficients  are 
actually  constant. 

We  now  give  a  characteristic  result  for  this  type  of  process  when 
higher  order  dependency  is  measured  by  Corr [Rt^  P  ^ .  ( 

CorrCRt^ ^ ^ ( At-t-r^ '  assuming  that  the  Rt^P^’s  are  uncorrelated. 


With  th«  raindom  coefficient  model  (4.1), 


CorrCRt^P^»(Xt-r“4)^l  Corr[Rt^P^/( Rt-r^^^ J  “'®  equal  to  zero  for 

r-1 ,2 . 


Proof:  Using  the  definitions  (2.S)  and  (4.1). 


R^(  P  )  -  (  Xt-M  )-«i(  )- .  .  .  -«p(  Xfc-p-M ) 

■  ( ^ ^“®1 )*t-l  +  ( At^ ^ ^-«2 )*t-2  +  •••  +  (At^P^-ap)Xt-p 

+  (  l—oi .  .  .  “Op  )>A  .  (4.2) 

On  multiplying  (4.2)  by  (R^-r^^^)^  end  ta)cing  expectations,  thus  obtaining 
Cov[Rt(P\(Rt-r^P^  )^]/  we  have  a  sum  of  p  terms  given  by 


P 

C 


E((At(:i) 


'j 


)Xt-j(Rt-r<Ph2} 


(4.3) 


3-1 

and  a  last  term  involving  Which  is  clearly  zero.  Now  At^  ^  is 

independent  of  both  the  X^-j  and  (Rt-r^P^)^>  r*l,2,...,  which  may 

nevertheless  be  themselves  dependent.  Thus  the  jth  term  in  (4.3)  becomes 

E(A<3  )-aj  )E{Xt-j(Rt-r^^'>^)  *0  for  r-1, 2, . .  . , 

since  E(A(3))«aj.  This  completes  the  proof  which  clearly  includes  the 
first  cross  correlation  mentioned  in  the  theorem.  The  proof  highltghta  the 
fact  that  tt  is  the  independence  of  the  vector  of  coefficients 
^ . ,At^ P ),Bt)  on  previous  X^'s  uhich  creates  the  effect  in  this  type 


of  mode  I . 

The  results  of  the  theorem  can  be  used  to  help  validate  random  coef¬ 
ficient  autoregressive  models;  also  useful  in  this  respect  are  the  non¬ 
zero  higher  order  residual  crosscorrelations,  e.g.  for  positive  r 
Corr[Rt^P^,(Rt+r^*’^  )^]  •  These  have  been  obtained  for  the  second  order 
autoregressive  exponential  process  studied  in  Lawrance  and  Lewis  (1985); 
similar  results  for  amy  first  order  ramdom  coefficient  model  of  the  type 
(4.1)  aure  given  in  the  next  subsection. 

Xt  is  worth  noting  that  with  defined  ais  any  reasonable  function  of 

(X^_r,Xt-r-l' ■ • • »  similaur  argument  to  that  given  in  the  proof  shows  that 

Corr[Rt(P),Gt_r)-0,  r-1,2 .  (4.4) 

A  result  of  this  type  is  not,  however,  sufficient  to  establish,  for 


instanc*,  that  Rt^P)  and  Rt-r^P)  ara  indapandantf  thay  ara  dapandant. 

Tha  randon  coafficiant  autoragraasiva  atructura  of  (4.1)  ia  not  a 
nacaaaaxy  condition  for  tha  raaulta  of  thia  aaction  to  holdi  thia  may  be 
illuatratad  by  noting  that  CorrCR^^P),(Xt..r~<^)^]  zaro  idien 

Ci2(  Mtiafy  tha  aquationa 

Ci2(r)-aiCi2(r-l)+a2Ci2<r-2)+...+OpCi2(r-p),  r-1,2 . (4.5) 

Tha  aimilarity  of  thaaa  aquationa  to  atandazd  Yula-WaUcar  aquationa  (2.4) 
will  ba  apparant.  Thua  any  procaaa  with  thia  property  will  have 
Corr[Rt^PK(Xt_r-M'*]  equal  to  zero  for  non-nagatlva  r. 

4.2  Higher  Order  Dapandancy  for  Plzat  Order  Random  Coafficiant 
Jtoitorayiaaaiaa  Wadala 

The  model  to  be  conaidazad  ia  tha  firat  order  (p>l)  caae  of  (4.1),  now 
to  be  denoted  aa 

Xt-RtXt-l+®t»  t-0,  tl,t2, . . .  ,  (4.6) 

in  v«hich  and  Bt  are  independent  bet%4aen  each  t,  but  within  themselves 
may  be  dependent)  v  alao  %iarite 

a-E(At),  M-E(Xt)-(l-*)”^E(Bt). 

Many  basic  mathematical  and  probabilistic  properties  of  this  equation  have 
been  studied  by  Vervaat  (1979).  Interest  here  is  restricted  mainly  to  the 
residual  crosscovariances  of  (Rt^,Kt-r)  ^nd  C(X^-4)^,Rt-r]  R^  is  the 

first  order  residual  given  by 

Rt-Xt-)x-a(  Xt_i-4 ) .  (4.7) 

It  has  been  established  in  Section  4.1  that  Cov(R^,R^_f2^^ov(R^^,R^4.f )>0 
for  r-1,2,...,  and  so  now  Cov(  R^2  ^  ^  fo,.  r^,l,2,...  is  obtained 

explicitly. 

The  calculation  begins  by  vrriting  the  required  covariance  as 
Cov(  Rt  2 ,  Rt-r  )-E  ( [  ( Xt-n  )-a(  )  ]  2  ( ( Xt_r-M  >-a(  Xt-r-l-<* )  1 1 

-C2i( r )-aC2i( r+1 )-2a(Ciii( r )-aCiii(  r+1 ) ) 

+a2{C2i(r-l)-aC2i(r))  (4.8) 

where,  as  defined  just  before  (4.5), 

C2i(  r  )-E  { ( Xt-M  )2(  Xt-t-n ) } ,  Ciii(  r  )-E  ( ( XfH )( Xt_i-M )( Xt_r-»i ) ) . 


The  higher  order  triple  moment  Cxii(r)  is  easily  seen  to  be  equivalently 
given  by  aC2i(r-l},  in  the  present  instance,  and  hence  (4.S)  becomes. 


Cov( ,  Rt-r )“C2l( r )-aC2i( r+1 )-a2 {C2i( r-1 )-aC2i( r ) ) ,  r-1 ,2, .. .  (4.9) 


For  r^,  with  43-E[(Xt->i)^] ,  there  is  the  seperate  result 

E(  Rt^ )-( i+2a3  )43-3aC2i(  1 ) . 


(4.10) 


The  calculation  of  C2i(r)  is  effected  by  «n:iting  the  defining  equation 
(4.6)  in  the  form 

Squaring  both  sides,  multiplying  by  snd  taking  expectations  then 


gives 


where 


C2i(  r  )-a2C2i(  r-1  )+2(  )-na  )C(  r-1 ) 


C(  r  )sCov(  Xt ,  Xt-r  Xt ) ,  a2*E(At^)' 

Further  simplifications  of  (4.11)  give  the  recursive  equation 

C2l(  r  )-S2C2l(  )+Xa*‘“l ,  r-1 , 2 ,  .  .  . , 


where 


k-2<T2  { Mvar(  At  )+Cov(  At ,  Bt ) ) . 


(4.11) 


(4.12) 


Equation  (4.12)  has  ei^licit  solution 


C2i(  r  )-a2*U3+k(  a2^-a^^ )/( a2-a ) ,  ( a2Sa ) 

r-0,1, 

-a*43+Jsra’^'^  -  ( a2"* ) 


(4.13) 


Going  back  to  (4.9)  for  Cov(Rt^,Rt-r )  and  using  (4.12),  we  have  finally 
Cov(  Rt^  *  Rt-r )•( a24a3a2-aa2^-a2 )C2i( r-1 ) 

+( l-a2+a3-aa2  )ka>^“l ,  r-1 , 2 ,  .  .  .  .  (4.14) 

Thus,  (4.14)  and  (4.13)  constitute  the  explicit  solution  for  the 
crosscovariances  of  the  residuals  (Rt^,Rt-r)’  notice  that  when  At  is  the 
constant  a,  the  case  of  the  standard  first  order  linear  model,  these 
crosscovariances  are  correctly  zero.  The  NEAR(l)  model  of  Lawrance  and 
Lewis  (1981)  specializes  (4.6),  by  having  independence  within  each  (At.Bt) 
and  the  particular  forms 

(0  w.p.  a  a  .  »  V  '^-P-  P 

^  to  w.p,  1-a'  ^  ^^t^t'  *^t  2^_p 


where 


b  -  (1-0)0,  p  -  (1-0 )/{!-( 1-0)0) 


and  (Rt)  independently  distributed  exponential  sequence  with 
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paraawt«r  x. 


If  th«  covariances  (4.14)  are  to  bs  converted  to  correlations,  then 

var(Rt)  Var(Rt*)  inust  be  obtained.  This  requires  the  folloiifing 

calculation, 

Var(Rt*)  -  E<Rt*>  -  CE(Rt*)l*  (4.15) 

%dtere 

E(Rfc*)  -  (1  -  a*)<y*  -  Var(Rt),  (4.16) 

E(Rt*)  -  (l+a*)*i4  -  4aC3i(l)  +  6a*C22(l)  “  4a»Ci3(l),  (4.17) 

with 

ji4-EC(Xt-Ji)^l,  Cij(l)-ECXt-u)i(Xt-i-H)i],  (i,j)-(3,l)/(2,2),(l,3).(4.18) 
The  joint  moments  in  (4.17)  are  now  obtained  in  terms  of  the  first  four 
moments  about  the  mean  of  X^,  together  with  the  first  three  moments  and 
joint  moments  about  zero  of  and  Thus  in  terms  of  the  further 

quantities 

ag  -  E(At*),  b2  -  E(Bt*),  dij  -  E(AiBj),  (  i,  j  )-( 1, 1 ),  ( 1, 2  ),  (  2, 1 )  (4.19) 


we  have,  as  with  the  derivation  of  (4.11),  that 

^31<  1 )  ■  3|A*c*a“3ji(  M3+2J1C*  )a2+(  *i4+34H3+3(**0’*  )A3 

“dtur^di  T  *3<r*dT  yt3(  )^Z1  •  (4.20) 

C22(l)  ■  -4*0’*-2ftM3a+(  J14+2M43+H*<7*  >42 

+C^b2+2(43+J«y*)<Jll.  (4.21) 

Ci3(l)  -  44a-  (4.22) 

The  results  ( 4. 20 )-( 4.22 )  have  been  chec)ced  against  an  alternative  method 
of  calculation  involving  moments  about  zero  instead  of  about  4. 

It  was  remarked  in  Section  3,  following  (3.2),  that  the  cross¬ 
correlations  of  (Xt-4)‘  and  can  also  be  useful  in  assessing  higher 

order  dependence.  In  the  present  case  of  first  order  residuals  and 
autoregression  there  are  the  results, 

Cov[(Xt-4)*,  Rt-r]  ■  C2i(r)  -  a  C2i(r+1),  (4.23) 

var(Rt)  -  (l-a«)<T*,  var((Xt-4)*)]  -  44  -  »*•  (4.24) 

These  formula  can  all  be  applied  to  the  NEAR(  1 )  model  of  unit  mean  by 
simply  noting  that  in  this  case 
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E(At’^)  -  afl'.  E(Bt')  -  rlClH-(l-p)b5^]. 


(4.25) 


PiguTM  1  and  2  gi.v«  tha  raaxtlta  of  computations  of  CorrCRt*'I^t-r] 
CorrC(Xt-(i)*>Rt-r]  NEAR(l)  nodal.  Thm  top  left  frama  of  Figura  l  is 

tha  linaar  EAR( 1 )  caaa  for  which  all  cross-correlations  apart  from  lag  zero 
are  zero;  tha  other  three  cases  in  Figura  1  each  have  zero 

crosscorralations  at  negative  lags,  in  agreement  with  the  theoretical 
results  in  Section  4.1,  but  have  some  non— zero  values  at  the  zero  and 
positive  lags.  It  is  evident  that  the  lag  zero  cross  correlations  contain 


a«0  999.  ^■0.736:  Cas* 


of 

* 

qT 


O 

■A 

A 

O 

'3 


A 

o 


o 


a-0  910.  ^-0  $2* 


Figur*  1.  Four  computations  of  tha  crosscorralations ,  for  various  lags, 
batuaan  tha  linaar  autoragrassiva  rasidual,  Rf(P)  and 


ffSAR( 1 )  procass  uith  p{l)~a0  hald  constant  at  0.75;  in  affact  tha  remaining 
free  paranatar  is  baing  varied  through  its  allouabla  range. 


nuch  of  th«  discrininatlng  information  botwoon  tho  four  casos;  this  will  bs 
so  for  first  ordar  autorsgrassiv*  modsls  in  which  much  of  ths  highsr  ordsr 
dspsndsncy  is  at  lag  ons,  and  tha  lag  zaro  cross  corralation  involves  both 
third  and  fourth  ordar  aspects  of  the  lag  one  dependency.  For  further 
ii4£ozmation  on  the  TEhR(  1 )  and  ?R£AR(  1 )  cases  see  Lawrance  and  Lewis 
(1981). 

Figure  2  gives  CorrC(X^-M)^<Rt-rl  same  four  cases  used  in 
Figure  1.  The  negative  lags  are  again  zero  for  all  cases,  and  hence  this 
property  does  not  discriminate  the  linear  from  the  nonlinear  cases.  Rather 
it  is  the  strength  of  the  crosscorrelations  at  positive  lags  which  performs 
this  task,  albeit  less  clearly  than  the  crosscorrelations  of  the  residuals 


amd  squared  residuals. 


a-a.399.  i-0  ^  Can 


a«0  3>0.  i«0  SI* 


NEAR(  1 )  process  <jith  p(i ;«a(3  held  constant  at  0.75/  in  effect  the  remaining 
free  parameter  is  being  varied  through  its  aliouable  range. 


for  tho  Product  AntozogroMioo  Modol 


Anothor  fora  of  nonlinour  axitotmqrmuaivm  nodol,  callod  PAR(l),  was 


introducad  hr  NcICancia  (198f)i  tha  basic  Idaa  of  Its  construction  is  to 


considar  tha  axponantiation  of  tha  standard  AR(1)  aquation.  nian  its 


addltiva  structura  bacosMs  multlpUcativa,  and  tha  ganaral  form  of  tha 


PAR(l)  product  autoxagrassiaa  nodal  nodal  is 


(4.26) 


«^ara  p  is  tha  dapandancy  paramatar,  0<p<l,  and  (Bt)  is  eui  indapandant  and 


Idantically  dlstributad  innovation  saquenca,  indapandant  of  Xt-i/Xt-2. . . .  . 


McKanzia  studlas  tha  nodal  whan  is  chosan  so  that  (X^)  has  a  gamma 


marginal  distribution,  and  obtains  savazal  basic  resxilts;  for  instemca, 


that  p  still  raprasants  tha  lag  one  autocorrelation  auid  that  tha 


autocorrelations  in  ganaral  satisfy  the  Yule-WalXar  first  order  aquations 


((2.4)  with  p-1),  so  that  p<r)-p’f.  It  is  apparent,  however,  that  the 


linear  conditional  aspactation  definition  of  autoregression  (equation 


(2.4))  is  not  satisfied  because  of  tha  power  form  implied  by  (4.26). 


As  with  tha  first  order  random  coefficient  models  in  Section  4.2,  we 


consider  tha  first  order  residual  R^,  given  by  (4.7),  atui  will  likewise 


determine  Cov(R^2^j^_^)  x*0,  ti,  ±2, . . . ,  noting  that  for  this  model  mesa 


correlations  are  non-zero  for  all  lags.  We  will  use  the  general  expression 


(4.9)  in  terms  of  the  third  order  central  moments  C2i(r)  and  Ciii(r),  but 


this  time  there  is  no  simple  relation  between  them,  and  both  are  needed  for 


all  lags.  Also,  calculation  of  C2i(r)  and  Cj,x2^(r)  must  be  in  terms  of 


their  uncentared  components,  since  these  are  the  quantities  which  can 


immediately  be  determined  from  the  PAR(1)  equation  (4.26).  The  required 


uncentered  moments  will  be  written  as 


a2-E(Xt2),  e3-E(Xt3) 

42 i( r )-E( Xt^Xt-r ) /  aiii( r )-E( XtXt-iXt-r ) 


(4.27) 


for  r>0, ti, s2, . . . ,  and  there  is  need  to  note  the  special  cases. 


a2i(0)aa3,  aiii( 0  )se2i(  1 ),  •in(  1  )“«2l(“l  )> 


All  th«M  quantities  trill  ba  calculated. 


First,  it  is  necessary  to  note  the  foUovinq  relations  between  the 
centred  and  uncentred  moments, 

C2i(  r  )“*2l(  ^  )-n^-2tiC(  r  ( all  ^ ) 

Clll<0)-C2i(l),  Ciii(l)-C2lC-l>  (4.28) 

Ciii( r )-eiii( r )-u3-m(C<  r )4C( r-l )4C( 1 ) ) ,  ( n*0, 1) , 

The  calculations  of  e2i(r)  and  eiii(r)  need  to  be  treated  separately 
for  positive  and  negative  laqs,  but  follow  in  the  same  general  manner  and 
will  be  illustrated  by  that  for  e2i(r)  for  positive  lags.  By  iterating  the 
PAR(1)  equation  (4.26)  r  steps  bac)cward, 

r-l 

Xt  -  Xt-r^'>  n 
i-0 

where  p< r )ep',r-0, 1, . . . .  Squaring  this  equation,  multiplying  it  by  Xt-r 
and  taking  ejq>ectatlons ,  gives 

r— I 

•2l(r)-E{Xt-r^^^'^'*’^)  n  E(Bt-i2'=*<^>).  (4.29) 

i-0 

To  obtain  the  expectations  in  the  repeated  product,  ta)cing  the  2p(i)th 
power  of  (4.26),  leads  to 

E(Xt*^  ^ ^  ^  )Ef ^ , 
and  hence,  dropping  the  unnecessary  suffix  t,  to  the  result 
E{b2P(  i))-E{x2f»(^))/E(x2P(  i+D). 

Now  (4.29)  can  be  expressed  purely  in  moments  of  X,  as 

e2i(r)-E{x2P(  J^)+l)E(x2)/E(x2P(r)) .  (4.30) 

To  proceed  further,  invoke  a  gamma  marginal  distribution  for  X  with  density 
f(x)-«^*x^le“^/r(0),  e,0>O,  x>0»  (4.31) 

this  has  mean  0/B,  variance  0/6*,  third  central  moment  20/ ,  and  there  is 
the  kth  moment  result  E( ^ )-r( 0+h )/e^r( 0 ) .  Prom  (4.30)  and  similar 

expressions  we  then  have 


e2i(r) 


•lll(r) 


r0(O4-l)(O4-2p»^)/e5, 
b(0+pH^I  )(<HpH^'+l  )/63, 


0(  0+P )( 0+P*^^+P^  )/9^ . 
./3((HP*’^'  )(<Hp+p'*‘''^^)/©^, 


r-l , 2 , . . . 
r— 1,-2, . . . 

r-2,3, . . . 
r— 1,-2 _ 


(4.32) 


(4.33) 


with  *21(0),  •iii(O)  «nd  •mCD  b«ing  givsn  by  the  special  cases  of 
(4.27).  Ose  of  (4.28)  gives  finally  the  required  expressions 


C21<r)  -  {Jf^liT!p2lrl)/e3, 
Clll(r>  -  ff5olr?+l4.i2?ri- 


r-0, 1, . . . 
r— 1,-2, .  .  . 


(4.34) 


3 

U{plrl+l+p2lr|+lj/e3  r— 1,-2* ! . . 

All  these  may  be  used  in  (4.8)  to  obtain  the  desired  result  for 
Cov(Rt*,Rt-r)  “ 


Cov(Rt*,Rt-r)  - 


f20( 1-p* r-1 , 2 , . . . 

0( 2-6p^+p»+3p* )/©»  r-0 

-0(  1-p )( 1-p^  )*p^  <  r  I  -l/e*  r— 1 ,  -2 , .  .  . 


(4.36) 


The  simpler  covariance  of  (Xt-4)‘  and  Rt-r  Also  follows  via  (4.23)  and 
(4.35)  as 


_  r20(  1-p*  )pJ^/©» 


co,((xt-4)^m.,i  -  i:-;— <■•■”> 

Conversion  of  these  covariances  to  correlations  requires  Var(Rt*)  ets  at 
(4.15)  and  Var[(X^-u)*]  as  at  (4.24);  the  required  intermediate  results  are 

M4  -  30(0+2)/6*, 

Caid)  -  3d(/3+2)p/©*,  Ci3(l)  -  0{304.(l+p)(2+p)}p/©*, 

C22<1)  -  (0*+2)3p+20(/3+2)p*)/©*.  (4.38) 

These  then  give  as  at  ( 4.15 )-( 4. 18 )  the  explicit  expression 


Var(Rt*)  “  20{(0+3-2(/3+6)p*+6p»+(/3+ll)p*-6p»-2p*)/©*.  (4.39) 

Together  with  Var(Rt)  which  is  simply  (l-p*)0/9*,  Cov(  Rj.*  ,Rt_r )  can  be 
converted  into  Corr(Rt*,Rt-r)-  Similarly,  (4.37)  requires  the  result 

Var{(Xt-u)*}  -  20(0+3)/©*  (4.40) 

in  order  to  obtain  corr{(Xt-M)*»Rt) • 

Figures  3  and  4  illustrate  the  residual  and  squared  residual  cross 
correlations  for  the  PAR(l)  model.  Figure  3  shows  how  these  residual  cross 
correlations  vary  over  the  range  of  p  values  from  p-0  to  p-0.9  for  an 
exponential  marginal  distribution.  At  p~0,  the  PAR(l)  is  an  IDO  process 
and  the  residuals  are  trivially  independent,  resulting  in  the  only  non-zero 
crosscorrelation  at  lag  zero.  As  the  p  value  increases  the  dependence 


spreads  out,  mosc  strongly  at  positive  lags.  An  interesting  feature  is  the 


lag  croaacorralation  which  changas  from  baing  strongly  poaitiva  at  p-0 

to  approximataly  xaro  at  p-0. 75,  to  nodarataly  nagativa  at  p-0.9 


MM.  ,-a.oeo.  i-i.o  <«i.ooo.  ,^100.  •-1.0 


Figure  3.  Computations  of  the  residual  crosscorrelations,  CorrCR^^ 
for  the  PAJi(l)  model.  Since  /3-i  the  marginal  distribution  is  exponential . 
The  lag  one  serial  correlation  is  increased  from  p(l)^0.0  (upper  left)  to 
p(  1)^.9  (lover  right). 

Pigura  4  gives  four  different  garrnia  cases  of  the  PAJt(  1 }  residual  cross 
correlations,  all  with  p-0. 75.  The  gamma  shape  parameter  0  takes  the 
values  0.5,  1.0,  2.5  and  10.0;  these  cases  indicate  that  changes  in  the 
gamma  parameter  cause  only  slow  changes  in  detail  of  the  cross  correlations 
which  from  the  formulae  (4.37),  (4.39)  and  (4.40)  all  tend  to  zero  as  0^. 


;(ii 


i 


fl^o.soa.  p-0.750.  f>i.o 


tf-l  000.  P-0.750.  0 


d-2.S00.  P-0.7S0.  «-l  0 


/J- 10.000.  p-0.750.  »-'  .0 


FtgiLr0  4.  Computations  of  ths  rssidual  crosscorrslations ,  Cot’r(R'^^  i 

for  tf\0  PAJi(  1 )  model.  77>»  p  for  each  case  of  0.75,  and  tn»  figures 
illustrate  the  effect  on  the  residual  crosscorrelations  of  changing  the 
index  0  of  the  gamma  distribution  through  0.5,  1.0,  2.5  and  10. O. 


5 .  CONCLOSIONS 

A  methodology  for  analyzing  higher  order  dependence  in  nonlinear  time 

series  with  pth  order  autoregressive  correlation  structure  has  been 

proposed.  It  utilizes  standard  uncorrelated  linear  autoregressive 

residuals,  and  the  crosscorrelation  function  of  these  residuals  and  their 

squares.  The  behaviour  of  this  crosscorrelation  function  has  been  utilized 

for  two  rather  different  types  of  nonlinear  model:  random  coefficient 

autoregression  and  multiplicative  autoregres.sion :  the  behaviour  has  been 

20 


‘fMJ 


er». 


shown  to  allow  discrinlnation  b«t%#a«n  nodals  in  ths  saina  class  with  ths 
sains  marginal  and  autocorrslation  structurss. 

The  residuals  crosscorrelation  function  provides  a  partial  analysis  of 
third  order  information  in  the  time  series;  it  does  not  attempt  to  capture 
all  third  order  information,  %Aiich  is  the  aim  of  such  techniques  as 
bispectral  analysis  ^md  which  will  often  be  intractable  with  nonlinear 
models.  Being  based  on  stemdard  lineeur  residuals,  the  analysis  extends 
rather  than  replaces  conventional  residual  analysis. 

Developments  of  the  analysis  which  focus  on  the  directionality  implicit 
in  many  time  series  are  being  investigated;  reversed  residuals  assume  a 
reversed  directionality  and  allow  exploration  of  the  consequences  of  such 
an  asssumption. 
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